Real-Time Vision-Based Sign Language Bilateral Communication Device for Signers and Non-Signers using Convolutional Neural Network

The use of sign language is an important means communication for individuals with hearing and speech impairments, but barriers can still arise due to differences in grammatical rules across different languages. In effort address these barriers, this study aimed develop a real-time two-way device that uses image processing recognition systems translate two-handed Filipino Sign Language (FSL) gestures facial expressions into speech; the system recognize correspond specific words phrases. Specifically, researchers utilized Convolutional Neural Networks (CNNs) enhance speed accuracy device. also includes speech-to-text (STT) feature helps non-signers communicate deaf without relying on interpreter. study's results showed achieved 93% rate recognizing FSL using CNN, indicating it highly accurate. Additionally, performed real-time, overall average conversion time 1.84 2.74 seconds text, respectively. Finally, was well-received by both signers non-signers, total approval rating 85.50% from participants at Manila High School, suggesting effectively facilitates has potential break down barriers.